This invention relates generally to social networking, and in particular to training a content quality metric prediction model using polling results.
Online advertising has quickly become a major channel through which advertisers market their products and services. Traditional performance metrics are used primarily to determine the effectiveness of advertisements, measured for example by click through and conversion rates. These traditional performance metrics do not evaluate the quality of an advertisement, including the content of the advertisement. Hence, the question of whether users actually enjoyed an advertisement remains unanswered by these performance metrics.
One problem of measuring and predicting the quality of advertising stems from the inherently subjective nature of audiovisual content and people's reactions to it. This is illustrated by the example of a music competition in which three judges rate, on a scale from 1 to 10, the performance of a singer without being able to judge the singer's visual performance. In this hypothetical, the judges are tasked with quantitatively scoring the singer based on the singer's vocal ability and musical performance. Each of the judges may arrive at different scores due to their relative scoring biases. One judge may be very generous in the scoring and only delineate between singers on a much smaller scale than another judge who may be very harsh in his scoring. Regardless, an absolute winner of the competition is eventually determined by normalizing the scores. But the judges' scores are only applicable to the specific singers in the competition, not all singers in the world.
Similar to the singing competition hypothetical, advertising models have relied on the effectiveness of an advertisement, such as asking a focus group how likely they are to buy the product mentioned in the advertisement in the next 6 months, on a scale of 1 to 5. Advertisers and publishers have no way to determine, based on the data gathered, which advertisements are of higher quality and which advertisements are of lower quality. Based on the data gathered, advertisers may only extrapolate on a small sample size the subjective opinions of focus group members. Advertisers rely on the focus groups to determine if the advertisement will be effective, not whether the people in the focus groups actually enjoyed the advertisement.
Attractive advertisements tend to increase engagement with the advertiser's brand, leading to more user traffic on the publisher's website and an increase in the overall advertisement fees collected by the online services. Social networking systems have also enabled advertisers to let users share interesting advertisements with their connections on the social networking system, creating “viral” advertising. This “word of mouth” advertising is difficult to generate because advertisers and publishers do not have an accurate sense of what advertisements are enjoyable, and, consequently, more likely to go viral.
To take advantage of the millions of users that use social networking systems, advertisers need better metrics on the content of their advertisements. Publishers of advertisements have not created tools or techniques for advertisers to receive feedback on the quality of their advertisements with respect to the user experience of the content within the advertisements. Tools and methods are needed to address this problem of determining a discrete measurement of content quality.